Generation of implicit tv recommender via shows image content

ABSTRACT

A system and method for providing recommendation according to the preferences of the user based on the user&#39;s past viewing history. The system keeps track of the program image contents watched by a user to generate a user&#39;s preference. Then, a plurality of recommendations based on the viewer&#39;s viewing habit stored in the user profile is offered to the user, so that the user can selectively watch the programs of his or her interest.

The present invention relates to a system and method for recommendingtelevision programming and, more particular, to a system and method forproviding a particular programming of interest based on an imagerecommendation made from the customer's preference or profile.

The modern world with huge amounts of multimedia gives televisionviewers a tremendous variety and range of options. Currently, there areover 500 different program channels shown through a cable televisionservice. As the number of channels available to television viewers hasincreased, along with the diversity of programming content available onsuch channels, it has become increasingly challenging for televisionviewers to identify television programs or retrieve information of theirinterests.

There are different types of television program guides that areavailable, i.e., the electronic program guides (EPGs), which allowtelevision viewers to sort or search the available television programsin accordance with personalized preferences. The EPGs allow viewers toidentify desirable programs more efficiently than conventional printedguides. As many viewers have a particular preference for or bias againstcertain categories of programming, such as the viewer's preferences, canbe applied to the EPG to obtain a set of recommended programs that maybe of interest to a particular viewer.

These TV show recommenders help users better manage the tremendousnumber of choices. However, users do not always trust that theserecommenders work well or that they even have the user's best intentionsin mind. Accordingly, the present invention provides a novel way ofincreasing trust in a recommender by having it presenting itself asbeing “on the same side” as the user and by reminding the user of arelated show or event observed by both the user and the recommender inthe past.

The present invention is directed to a method and system for providingcontent recommendations according to the past viewing history of aparticular topic.

According to one aspect of the invention, a method of providing arecommendation includes the steps of: receiving a plurality of programsfrom a plurality of channels; extracting key frames and content of theprograms watched by a user to determine preferences of the user;generating a user profile based on the extracted key frames and contentof the programs; and, providing a plurality of viewing recommendationsbased on the user profile. The key frames of a particular program isperformed interactively in response to the user's input, and the contentof the user profile is updated to reflect said user's changingpreference.

According to another aspect of the invention, a method of providing arecommendation includes the steps of: generating at least one userprofile indicating preferences of a user for a particular program basedon the video image contents and its electronic program guide (EPG) dataembedded therein; comparing the user profile with incoming programs froma plurality of channels to determine a desired program; and, if there ismatch, recommending the desired program to the viewer.

According to another aspect of the invention, a system of providing arecommendation includes an input device for providing viewing preferenceof a user; and a processor for: receiving a plurality of programs from aplurality of channels; extracting key frames and content of the programswatched by a user to determine preferences of the user; generating auser profile based on the extracted key frames and content of theprograms; and, providing a plurality of viewing recommendations based onthe user profile.

According to yet another aspect of the invention, a system of providinga recommendation includes an input device for providing viewingpreference of a user, and a processor for: generating at least one userprofile indicating preferences of a user for a particular program basedon video image contents and its electronic program guide (EPG) dataembedded therein; comparing the user profile with incoming programs froma plurality of channels to determine a desired program; and, if there ismatch, recommending the desired program to the viewer.

FIG. 1 illustrates a simplified block diagram of the system according toan embodiment of the present invention.

FIG. 2 shows major components of the system according to an embodimentof the present invention.

FIG. 3 is a flow chart illustrating a user profile development sessionin accordance with the present invention.

In the following description, for purposes of explanation rather thanlimitation, specific details are set forth such as the particulararchitecture, interfaces, techniques, etc., in order to provide athorough understanding of the present invention. However, it will beapparent to those skilled in the art that the present invention may bepracticed in other embodiments, which depart from these specificdetails. Moreover, for the purpose of clarity, detailed descriptions ofwell-known devices, circuits, and methods are omitted so as not toobscure the description of the present invention with unnecessarydetail.

FIG. 1 shows a conventional television system 2 and a receiver 10whereto the preferred embodiments of the present invention may beapplied. The receiver 10 may also include other personal recordingdevices, such as ReplayTV (trademark of REPLAY NETWORKS, INC., of PaloAlto, Calif.) and TiVo (trademark of TIVO, Inc., of Sunnyvale, Calif.),that give the television viewer new abilities to capture and manipulatethe stream of television shows, which flow from their cable andsatellite systems.

As shown in FIG. 1, the receiver 10 is configured to receive audio andvideo programming from the conventional television (TV) broadcast aswell as other variety of sources, including a cable service provider,digital high definition television (HDTV) and/or digital standarddefinition television (SDTV) signals, a satellite dish, a conventionalRF broadcast, an Internet connection, or another storage device, such asa VHS player or DVD player. The audio and video programming can bedelivered in analog, digital, or digitally compressed formats via anytransmission means, including satellite, cable, wire, televisionbroadcast, or sent via the Web. It should be noted that the presentsystem is also capable of being connected to other possible networks,such as a direct private network and a wireless network.

According to the teachings of the present invention, viewers can berecommended with shows of their interest the receiver 10 based on thepast viewing history, preference, or user profile. To achieve this, thereceiver 10 detects incoming television programs from a plurality ofdifferent channels and maintains a database containing the viewinghistory of a particular viewer based on the show's video images and/orany optional EPG-related information, then determines the type of TVprograms that are watched or not watched by the viewer. Note thatreceiver 10 may comprise a separate channel can be dedicated to receivethe electronic program guide (EPG) data from a different source.Thereafter, a recommendation as to the type of programs or channels thatthe viewer likes to watch is generated according to the past viewinghistory.

Now, a detailed description of the present invention will be explainedhereinafter.

FIG. 2 illustrates an exemplary receiver 10 in greater detail accordingto the embodiment of the present invention. The receiver 10 includes aninput interface (i.e., IR sensor) 12, MPEG-2 encoder 14, hard disk drive16, MPEG-2 decoder 18, controller 20, feature extractor 22, videoprocessor 24, memory 26, and playback section 28. It should be notedthat the MPEG encoder/decoder may comply with other MPEG standards,i.e., MPEG-1, MPEG-2, MPEG-4, and MPEG-7. The controller 20 oversees theoverall operation of the receiver 10, including a feature extractionmode, record mode, play mode, and other modes that are common in atypical receiving device. It should be noted that the receiver 10 may beembodied in the television system 2.

During a normal mode, the controller 20 causes the incoming televisionsignals to be demodulated and processed by the video processor 24 andtransmits them to the television set 2 for display. Then, the videoprocessor 24 converts the incoming TV signals to the correspondingbaseband television signals that are suitable for display on thetelevision set 2.

During a recording mode, the controller 20 causes the MPEG-2 encoder 14to receive incoming television signals delivered from satellite, cable,wire, and television broadcasts or the web and converts the received TVsignals to the MPEG format for storage on the hard disk driver 16.Thereafter, the controller 20 causes the hard disk driver 16 to streamthe stored television signals to the MPEG-2 decoder, which in turntransmits the decoded TV signals to be transmitted to the television set2 via the play back section 28 for display.

During a feature extraction mode, the feature extractor 22 operates toprocess and generate data that is representative of the viewing historyof a given user to determine the type of programs most watched by theviewer, then generates the user's viewing habits to build a user profilein the memory 26. Alternatively, the viewer may explicitly designate thetype of shows or programs he or she prefers when generating the userprofile.

To build the user profile, video information is input to a featureextractor 22 where the content of the video information is analyzedaccording to the type of (1) show image content and (2) a pre-setclassification. A well-known electronic program guide (EPG) protocol maybe utilized for the provision of a pre-set classification. An EPG is astandard application designed to aid the viewer in the navigation of andselection from a broadcast material available in a TV environment. TheEPG provides information about each program and includes programmingcharacteristics, such as the channel number, program title, start time,end time, elapsed time, time remaining, rating (if available), topic,theme, and a brief description of the program's content. Using the EPGfeatures, the feature extractor 22 classifies the incoming programs intoone of the pre-set categories, i.e., sports, old classic movies, actionmovies, etc.

In addition to the EPG features, video images are analyzed according tothe embodiment of the present invention to learn and recognize the showsof interest for a particular viewer by evaluating the visual datacharacteristics, including color, texture, objects, object motion, andscenes, etc. It is noted that EPG features do not ways provide enoughdata for a subsequent recommendation of type of exact show of viewer'sinterest.

Accordingly, the feature extractor 22, upon receiving incoming videosignals, extracts keyframes (thumbnail images or the like). This inputcan be used as the preferred categories of the user along with the EPGinformation.

Note that detecting keyframes according to the content of the videoinformation is well known in the art that can be performed in a varietyof ways. See for example, Video and image processing in multimediasystem, by Borko Furht, Stephen W. Smoliar, and Hongjiang Zhang, KluwerAcademia Publishers, 1995, the content of which is hereby incorporatedby reference. Briefly, a video sequence comprised of a number of camerashots or scene changes go under a detection of scene changes forextracting key frames. One approach of detecting key frames is based ona histogram comparison approach, in which a comparison of correspondingpixels or regions in successive frames is performed to generate ahistogram of intensity levels. Each histogram for a frame is compared toa histogram with a next frame since histogram varies from one videosource to another. Then, a threshold is selected to determine whether apixel or region has changed across successive frames. The key frame thatexceeds threshold is selected as the key frame. This process is repeatedfor all the frames in the scene and for each scene change. Finally, thekey frames from each scene change is collated and divided into blocks,and then features, such as mean, standard deviation, entropy, anddominant colors are extracted. In an alternate embodiment, viewers canspecify the beginning and ending segment of a video sequence to go underthe feature extraction process, or can provide specific camera shots orkey frames via the input interface 12 of the receiver 10.

The video features extracted by the feature extractor 22 as describedabove, combined with the EPG information are provided to the memory 26to configure the types of video program shows of the viewer's interest.The content of extracted video key frames and key frames from thefeature extractor 22 are provided to the user profile of the memory 16to build a personal viewing preference. This viewing preference toparticular programs is then used to recommend the viewer during asubsequent viewing mode. In this regard, well-understood leaningparadigms, such as Hidden Markov Models (HMMs), Bayesian networks,genetic algorithms, and neural networks, can be employed to create auser profile. For example, U.S. patent application Ser. No. 09/466,406filed on Dec. 17, 1999 by the inventor of this application, entitled“method and Apparatus for Recommending Television Programming usingDecision Trees”, the content of which is hereby incorporated byreference.

Note that other learning models, or combinations thereof apparent to oneof ordinary skill in the art, may be used as well.

FIG. 3 shows user profile development steps according to the embodimentof the present invention. The receiver 10 observes the user's viewinghabits over time (step 100), during which each incoming video program isclassified as described above (step 120). Here, the user's viewingpreference, including the type of shows most frequently watched, andthis information is used to build a user profile in the memory 26.Thereafter, the user profile is updated (step 140). Accordingly, adatabase reflecting the user's likes or dislikes of various shows can beobtained based on the user's viewing pattern, such that the receiver 10can later recommend a list of programs that the viewer prefer based onthe past viewing history.

While the preferred embodiments of the present invention have beenillustrated and described, it will be understood by those skilled in theart that various changes and modifications may be made, and equivalentsmay be substituted for elements thereof without departing from the truescope of the present invention. In addition, many modifications may bemade to adapt to a particular situation and the teaching of the presentinvention without departing from the central scope. Therefore, it isintended that the present invention not be limited to the particularembodiment disclosed as the best mode contemplated for carrying out thepresent invention, but that the present invention include allembodiments falling within the scope of the appended claims.

1. A method for providing a recommendation, the method comprising thesteps of: receiving a plurality of programs from a plurality ofchannels; extracting key frames and content of said plurality ofprograms watched by a user to determine preferences of said user;generating a user profile based on said extracted key frames and contentof said plurality of programs; and, providing a plurality of viewingrecommendations based on said user profile.
 2. The method of claim 1,further comprising the step of selecting at least one viewingrecommendation by said user.
 3. The method of claim 1, furthercomprising the step of displaying at least one viewing recommendation.4. The method of claim 1, wherein the step of extracting the key framesof a particular program is performed interactively in response to saiduser's input.
 5. The method of claim 1, further comprising the step ofupdating the content of said user profile to reflect said user'schanging preference.
 6. The method of claim 1, wherein the step ofextracting the content of said plurality of programs is performedaccording to an electronic program guide (EPG) data.
 7. A method forproviding a recommendation, the method comprising the steps of:generating at least one user profile indicating preferences of a userfor a particular program based on video image contents and itselectronic program guide (EPG) data embedded therein; comparing saiduser profile with incoming programs from a plurality of channels todetermine a desired program; and, if there is match, recommending saiddesired program to said viewer.
 8. The method of claim 7, furthercomprising the step of selecting at least one desired program forviewing at a particular time.
 9. The method of claim 8, furthercomprising the step of storing said selected program in a storage mediumfor a subsequent replay.
 10. The method of claim 8, wherein said step ofthe selection of said desired program is performed interactively inresponse to said user's input.
 11. The method of claim 7, furthercomprising the step of displaying said desired program for viewing. 12.The method of claim 7, further comprising the step of updating thecontent of said at least one user profile to reflect said user'schanging preference.
 13. A system for providing a recommendation,comprising: an input device for providing viewing preference of a user;and a processor for: receiving a plurality of programs from a pluralityof channels; extracting key frames and content of said plurality ofprograms watched by a user to determine preferences of said user;generating a user profile based on said extracted key frames and contentof said plurality of programs; and, providing a plurality of viewingrecommendations based on said user profile.
 14. The system of claim 13,further comprising a memory for storing the preferences of said user.15. The system of claim 13, wherein the processor is further operativeto update the content of said user profile to reflect said user'schanging preference.
 16. A system for providing a recommendation,comprising: an input device for providing viewing preference of a user;and a processor for: generating at least one user profile indicatingpreferences of a user for a particular program based on video imagecontents and its electronic program guide (EPG) data embedded therein;comparing said user profile with incoming programs from a plurality ofchannels to determine a desired program; and, if there is match,recommending said desired program to said viewer.
 17. The system ofclaim 16, further comprising a memory for storing said at least one userprofile.
 18. The system of claim 16, further comprising a display fordisplaying said desired program for viewing.
 19. The system of claim 16,wherein the processor is further operative to update the content of saiduser profile to reflect said user's changing preference.